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FUNCTIONS 
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Abstract. Let iJ„ be the (2n + l)-diniensional Heisenberg group and K a compact group 
of automorphisms of i7„ such that {K k i7„, K) is a Gelfand pair. We prove that the Gelfand 
transform is a topological isomorphism between the space of iC-invariant Schwartz functions 
on Hn and the space of Schwartz function on a closed subset of M** homeomorphic to the 
Gelfand spectrum of the Banach algebra of A'-invariant integrable functions on i7„. 

1. Introduction 

A fundamental fact in harmonic analysis on is that the Fourier transform is a topological 
isomorphism of the Schwartz space iS(]R") onto itself. 

Various generalizations of this result for different classes of Lie groups exist in the lit- 
erature, in particular in the context of Gelfand pairs, where the operator-valued Fourier 
transform can be replaced by the scalar-valued spherical transform. Most notable is the 
case of a symmetric pair of the noncompact type, with Harish-Chandra's definition of a 
bi-ZT-invariant Schwartz space on the isometry group (cf. p. 489]). 

The definition of a Schwartz space on a Lie group becomes quite natural on a nilpotent 
group (say connected and simply connected). In that case one can define the Schwartz 
space by identifying with its Lie algebra via the exponential map. 

The image of the Schwartz space on the Heisenberg group if„ under the group Fourier 
transform has been described by D. Geller [TS]. Let Khe a. compact group of automorphisms 
of Hn such that convolution of i^-invariant functions is commutative, in other words assume 
that {K X Hni K) is a Gelfand pair . Then a scalar- valued spherical transform Qk (where 

2000 Mathematics Subject Classification. Primary: 43A80 ; Secondary: 22E25 . 
Key words and phrases. Gelfand pair, Schwartz space, Heisenberg group. 
Work partially supported by MIUR and GNAMPA . 



2 F. ASTENGO, B. DI BLASIO, F. RICCI 

Q stands for "Gelfand transform") of /^-invariant functions is available, and Geller's result 
can be translated into a characterization of the image under Qk of the space SxiHn) of 
i^'-invariant Schwartz functions. In the same spirit, a characterization of QxiSK^Hn)) is 
given in [5j for closed subgroups K of the unitary group \]{n). 

In [1] we have proved that, for K equal to U(n) or T" (i.e. for radial - resp. polyradial - 
functions), an analytically more significant description of QK{SK{Hn)) can be obtained by 
making use of natural homeomorphic embeddings of the Gelfand spectrum of L\{Hn) in 
Euclidean space. The result is that QxiSK^Hn)) is the space of restrictions to the Gelfand 
spectrum of the Schwartz functions on the ambient space. This condition of "extendibility 
to a Schwartz function on the ambient space" subsumes the rather technical condition on 
iterated differences in discrete parameters that are present in the previous characterizations. 

In this article we extend the result of pi] to general Gelfand pairs {K x Hn,K), with K 
a compact group of automorphisms of Some preliminary notions and facts are required 
before we can give a precise formulation of our main theorem. 

Let G be a connected Lie group and K a compact subgroup thereof such that {G, K) is 
a Gelfand pair and denote by L^{G/ / K) the convolution algebra of all bi-ii'-invariant inte- 
grable functions on G. The Gelfand spectrum of the commutative Banach algebra L^{G/ /K) 
may be identified with the set of bounded spherical functions with the compact-open topol- 
ogy. Spherical functions are characterized as the joint eigenf unctions of all G-invariant dif- 
ferential operators on G/K, normalized in the L°°-norm. G-invariant differential operators 
on G/K form a commutative algebra 3{G/K) which is finitely generated [T7]. 

Given a finite set of generators {Vi, . . . , Vg} of B){G/K), we can assign to each bounded 
spherical function (p the s-tuple ^(0) = (Vi{(l)), . . . , V^(0) j of its eigenvalues with respect 
to these generators. In this way, the Gelfand spectrum is identified with a closed subset 
Sj^ of C*. When all bounded spherical functions are of positive type and the operators Vj 
self-adjoint, T,^ C M*. As proved in [10], the Euclidean topology induced on T,^ coincides 
with the compact-open topology on the set of bounded spherical functions (see also [B] for 
G = K t< Hn and K C U(n)). When the Gelfand spectrum is identified with S^, the 
spherical transform will be denoted by Q^. 
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Let K he a. compact group of automorphisms of Hn such that [K x K) is a Gelfand 
pair and denote by the commutative algebra of left-invariant and i^-invariant differential 
operators on iJ„. Let V = {Vi, . . . , K} be a set of formally self-adjoint generators of D^^. 
We denote by iS(S^) the space of restrictions to of Schwartz functions on W, endowed 
with the quotient topology of S{W)/{f : f\ y = 0}. 

Our main result is the following: 

Theorem 1.1. The map Q\ is a topological isomorphism between SxiHn) and S(T,^). 

As customary, for us Hn is understood as M x C", with canonical coordinates of the first 
kind. It is well known that, under the action 

k ■ {t, z) = (t, kz) \/k G U(ra), (t, z) e 

U(n) is a maximal compact connected group of automorphisms of Hn, and that every compact 
connected group of automorphisms of Hn is conjugated to a subgroup of U{n). Therefore, if K 
is a compact group of automorphisms of Hn, then its identity component in K is conjugated 
to a subgroup of U(n). For most of this article, we deal with the case of K connected and 
contained in U(n), leaving the discussion of the general case to the last section. 

In Section [3] we show that it suffices to prove Theorem 1 . 1 for one particular set of genera- 



tors of 3k, and in Section |4] we choose a convenient set of generators. From a homogeneous 
Hilbert basis of i^'-invariant polynomials on M^" we derive by symmetrization d differential 
operators Vi, . . . , Vd, invariant under K; to these we add the central operator Vq = i~^dt, 
obtaining in this way a generating system of d + 1 homogeneous operators. Here we benefit 
from the deep study of the algebraic properties of the multiphcity-free actions of subgroups 
of U(n), developed in [21 IH El El 13 E] • In particular, rationality of the "generalized binomial 



coefficients" is a crucial point in our argument (see formula 7.5 below). It must be noticed 
that the proof of rationality in [8j is based on the actual classification of multiplicity-free 
actions. In this respect, our proof depends on the actual classification of the groups K giving 
rise to Gelfand pairs. 



After these preliminaries, we split the proof of Theorem 1.1 into two parts 
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In the first part we show that, if m is a Schwartz function on ]R'^+^, its restriction to is 



the Gelfand transform of a function / in iSx(if„) (see Theorem 5.5 below). The argument is 
based on Hulanicki's theorem [20] , stating that Schwartz functions on the real line operate on 
positive Rockland operators on graded nilpotent Lie groups producing convolution operators 
with Schwartz kernels. We adapt the argument in [27] to obtain a multivariate extension of 



Hulanicki's theorem (see Theorem 5.2 below). 

In the second part we prove that the Gelfand transform Q]^f of a function / in SxiHn 



can be extended to a Schwartz function on (see Theorem |7. l| below) . The proof begins 
with an extension to the Schwartz space of the Schwarz-Mather theorem [211 [2S] for C°° K- 



invariant functions (see Theorem 6.1 below). This allows us to extend to a Schwartz function 
on the restriction of Q^cf to the "degenerate part" Sq of the Gelfand spectrum (that 
corresponding to the one-dimensional representations of or equivalently corresponding 
to the eigenvalue for Vq). Then we associate to / a Schwartz jet on Sq. As in [1], the 
key tool here is the existence of "Taylor coefficients" at points of Sq, proved by Geller 



(see Theorem 7.2 below). The Whitney extension theorem (adapted to Schwartz jets in 
Proposition 7.4) gives therefore a Schwartz extension to W^'^^ of the jet associated to /. 
To conclude the proof, it remains to prove that if / G Sk^Hu) and the associated jet on 
So is trivial, then Q'^f admits a Schwartz extension. This is done by adapting an explicit 



interpolation formula already used in [T] (see Proposition 7.5). For a nonconnected group 
we remark that, calling Kq the connected component of the identity, one can view the K- 
Gelfand spectrum as the quotient of the ii'o-Gelfand spectrum under the action of the finite 
group F = K/Kq (it is known that if {KiKHn, K) is a Gelfand pair, so is [KoiKHn, Kq), cf. [2]). 
Starting from an F-invariant generating system of i^o^invariant differential operators, the 
iiT-Gelfand spectrum is then conveniently embedded in a Euclidean space by means of the 
Hilbert map associated to the action of F on the linear span of these generators. 

The paper is structured as follows. In Section [2] we recall some facts about Gelfand pairs 
(K X Hn, K) and the associated Gelfand transform. In Section [3] we show that it suffices 
to prove Theorem 1A_ for one particular set of generators of Dx- In Section |4] we choose a 
convenient set of generators in the case where K is a connected closed subgroup of U(n). 
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In Section [s] we show that every function in 5(M'^+^) gives rise to the Gelfand transform of 
a function in SK{Hn) via functional calculus. In Section [6] we extend the Schwarz-Mather 
theorem [2U [26] to Schwartz spaces. Section [7| is devoted to define a Schwartz extension on 
jjd+i q£ ^YiQ Gelfand transform of a function in Sx^Hn)- In Section [s] we show that our result 
holds for all compact groups of automorphisms of Hn- 

2. Preliminaries 
For the content of this section we refer to [H IH El [9l [TOl |22] . 

2.1. The Heisenberg group and its representations. We denote by Hn the Heisenberg 
group, i.e., the real manifold M x equipped with the group law 

{t, z) {u, w) = (t + u + ^Im w ■ z, z + w) t, M G M, ^z, w G C", 

where w ■ z is a short-hand writing for X]j=i "^j is easy to check that Lebesgue measure 
dtdz is a Haar measure on if„. 

Denote by Zj and Zj the complex left-invariant vector fields 

= — I Zj dt Zj = dzj + I Zj dt, 

and set T = dt- 

For A > 0, denote by J-'x the Fock space consisting of the entire functions F on such 
that 

equipped with the norm || ■ ||jc-^. Then if„ acts on JF^ through the unitary representation ttx 
defined by 

[7ix{t,z)F]{w) = e'^'e-^'"-'-i^'^' F{w + z) \/{z,t) e H^,, F e J^x, ^ G C" , 

and through its contragredient 7r_x{t,z) = 7Tx{—t,z). These are the Bargmann representa- 
tions of Hr,. 
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The space V{C"') of polynomials on C" is dense in J-'x (A > 0) and an orthonormal basis 
of 'P(C") seen as a subspace of J^x is given by the monomials 

^^•^^^^ = ((2/A)l-^ld!)V2 d ^ 
Besides the Bargmann representations, Hn has the one-dimensional representations 

with w G C". The ttx (A 7^ 0) and the fill up the unitary dual of Hn- 

2.2. Gelfand pairs {K x Hn, K). Let be a compact groups of automorphisms of Hn such 
that the convolution algebra L\{Hn) of integrable fC-invariant functions on if„ is abelian, 
i.e., such that {K x K) is a Gelfand pair. It is known [2j that this property holds for K 
if and only if it holds for its connected identity component Kq. On the other hand, every 
compact, connected group of automorphisms of Hn is conjugate, modulo an automorphism, 
to a subgroup of U(n), acting on Hn via 

k ■ {t,z) = {t,kz) \/{t,z)eHn, ke\]{n). 

For the remainder of this section, we assume that K is connected, contained in U(n), and 
{K K Hn, K) is a Gelfand pair. 

For the one-dimensional representation r^, we have T^{t,k^^z) = Ticw{t,z). Therefore, 
Tw{f) is a i^'-invariant function of w for every / G L\{Hn). 

As to the Bargmann representations, if A; G U(?2), 7r^;^(t, = T:±x{t,kz) is equivalent to 
7r±A for every A > and every choice of the ± sign. Precisely, we set 

v+{k)F{z) = F{k-^z) , u_{k)F{z) = F{k-^z) , 

each of the two actions being the contragredient of the other. We then have 

7ix{kz,t) = v+{k)'Kx{z,t)vj^{kY^ , TT_xikz,t) = u^{k)n_x{z,t)u_{k)~^ . 

By homogeneity, the decomposition of J-'x into irreducible invariant subspaces under i/+ 
(resp. z/_) is independent of A and can be reduced to the decomposition of the dense subspace 
P(C") of polynomials. 
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It is known since [9^, 3j that {K x K) is a Gelfand pair if and only if (equivalently 
decomposes into irreducibles without multiphcities (in other words, if and only if it is 
multiplicity free). The subgroups of U(ra) giving multiplicity free actions on P(C") have been 
classified by Kac [22] and the resulting Gelfand pairs {K x Hn, K) are listed in [SI tables 1 
and 2]. 

Under these assumptions, the space P(C'*) of polynomials on C" decomposes into 
irreducible subspaces, 

P(C") = 5^ P« , 

where A is an infinite subset of the unitary dual of and a denotes the equivalence class 
of the action on Pa. The irreducible i/_-invariant subspaces of 'P(C") are 

Pa' = {P{z) =p(^ -.pe Pa} , 

with the action of K on Pa' being equivalent to the contragredient a' of a. 

On the other hand, P(C") = XlmeN where ^^(C") is the space of homogeneous 
polynomials of degree m. Since h'± preserves each Vm{C"), each Pa is contained in Vm{C^) 
for some m. We then say that |a| = m, so that 

|a|=m |a|=m 

As proved in [3], all the bounded spherical functions are of positive type. Therefore there 
are two families of spherical functions. Those of the first family are 

Jk 

parametrized by iT-orbits in C" and associated with the one- dimensional representations of 
the Heisenberg group. 

The elements of the second family are parametrized by pairs (A, a) G M* x A. If A > 
and {vi, . . . , 'ydim(p„)} orthonormal basis of Pa in the norm of J-'x, we have the spherical 
function 

^ dim(P^) 

(2-1) <PxAt,z) = E Mt,z)v^,v^)^, . 
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Taking, as we can, Vj as A'^'^^f j, we find that 

0A,Q(2:,i) = (pi,ai'/>^z, Xt) . 

For A < 0, the analogous matrix entries of the contragredient representation give the 
spherical functions 

Setting for simphcity (pa = (pi,a, 

where qa G V{C") (S>P(C") is a real i^-invariant polynomial of degree 2\a\ in z and z (cf. [3]). 

Denote by 3k the algebra of left-invariant and i^-invariant differential operators on Hn- 
The symmetrization map establishes a linear bijection from the space of fT-invariant elements 
in the symmetric algebra over f)„ to Dj^. Therefore every element D G Vik can be expressed 
as Yl^jLo ^j'^'' 1 where Dj is the symmetrization of a ii'-invariant polynomial in Z,Z. 

Let D be the symmetrization of the /^-invariant polynomial P{Z, Z ,T). With D{(f)) 
denoting the eigenvalue of D G on the spherical function 0, we have 

(2.2) D{r]Kn^) = P{w,w,0) 



for the spherical functions associated to the one-dimensional representations 

For A 7^ 0, d7Tx{D) commutes with the action of K and therefore it preserves each P^. By 
Schur's lemma, dTrx{D)\p^ is a scalar operator cx,a{D) Ip^. It follows from (2.1) that 



In particular, for D = T, we have 



T{r]Kw) = , T[(f)x,a) = iX 
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3. Embeddings of the Gelfand spectrum 

Let K he a. compact group of automorphisms of if„ such that {K x if„, K) is a Gelfand 
pair. The Gelfand spectrum of the commutative Banach algebra L]^{H„) is the set of 
bounded spherical functions endowed with the compact-open topology. Given a set = 
{Vo, Vi, . . . , Vd} of formally self-adjoint generators of 3k, we assign to each spherical func- 
tion the {d + l)-tuple V{(f)) = ^Vo(0), Vi{(l)), . . . ,Vd{(f))j ■ Since dn{Vj) is formally self- 
adjoint for every irreducible representation tt , V{(j)) is in W^^^. It has been proved, in a more 
general context [lOj, that = {V{(f)) : (p spherical} is closed in W^^^ and homeomorphic to 
the Gelfand spectrum via V (see also [H]). 

Once we have identified the Gelfand spectrum with E^, the Gelfand transform of a function 
/ in L]^{Hn) can be defined on the closed subset of ]R'^+^ as 

J H„ 

In order to prove Theorem we first show that different choices of the generating 
system V give rise to natural isomorphisms among the corresponding restricted Schwartz 



spaces iS(E^). It will then suffice to prove Theorem 1.1 for one particular set of generators. 

On the Schwartz space iS(]R'") we consider the following family of norms, parametrized by 
a nonnegative integer p: 

||/||(p,M'")= sup (l+|y|fr/(y)|. 

j/eIR'",|a|<p 

Lemma 3.1. Let E and F he closed subsets o/M" and respectively. Let P : M" — > 
and Q : M™ be polynomial maps such that P{E) = F and Q o P is the identity on E. 

Given f in S{F) we let f = / o P\e- Then P^ maps S{F) in S{E) continuously. 

Proof. We show that if / is in iS(]R™), then P^ f can be extended to a function P'° f in iS(]R") 
in a linear and continuous way. Let \l/ be a smooth function on M" such that \I'(t) = 1 if 
\t\ < 1 and ^{t) = if |t| > 2. Define 



P^f{x) = m{x-Qo P{x)) (/ o P){x) Vx G W. 
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Clearly P^/ is smooth and -P^/Ib = P^f- Moreover P^f is zero when \x — Q o P{x)\ > 2, so 
it suffices to prove rapid decay for \x — Q o P{x)\ < 2. Note that there exists £ in N such 
that 

|a;| < 2 + |g(P(x))| < C(l + |P(x)|)^ Vx G M", \x - Q o P{x)\ < 2. 

Therefore given a positive integer p there exists a positive integer q such that ||P^/||(p,M'i) < 
C ||/||{g,iR"i)- The thesis follows immediately from the definition of the quotient topology on 
S{F) and S{E). □ 

Corollary 3.2. Suppose that {Vq, . . . , Vd} and {Wq, . . . , Ws} are two sets of formally self- 
adjoint generators o/D^. Then the spaces and 5(S^) are topologically isomorphic. 

Proof. There exist real polynomials pj, j = 0,1, ... ,s, and qt, h = 0,1, ... ,d, such that 
W, = pj{Vo, ...,Vd) and Vh = qh{Wo, . . . ,Ws). 

Setting P = [po, pi, . . . ,ps) : ^ M"+^ and Q = (go, Qi, ■ ■ ■ , Qd) ■ ^ R'^+\ we 



can apply Lemma [3 .l] in both directions. □ 

4. Choice of the generators 

In this section K shall be a closed connected subgroup of U{n) such that {K x Hn,K) 
is a Gelfand pair. The subject of the following lemma is the choice of a convenient set of 
formally self-adjoint generators of 3k. 

Lemma 4.1. A generating system {Vq = —iT, Vi, . . . , Vd} can be chosen such that, for each 
j = l,...,d, 

(1) Vj is homogeneous of even order 2mj; 

(2) Vj is formally self-adjoint and Vj{(f)a) is a positive integer for every a in A; 

(3) Vj{rixw) = Pj{w,'w), for every w in C", where pj is a nonnegative homogenous poly- 
nomial of degree 2mj, strictly positive outside of the origin. 

Notice that ([T]) and (|2| imply that when j = 1, . . . ,d 

(4.1) V'.^^A,^) = |A|"^ V,{<p^), VA G M \ {0}, Va G A. 
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Proof. Let CJ^ denote C" with the underlying structure of a real vector space. We denote 
by V{C^) = P(C") ® P(C") the algebra of polynomials in z and z, and by V^{C^) the 
subalgebra of i^'-invariant polynomials. 

The fact that the representation of K on P(C") is multiplicity free implies that the trivial 
representation is contained in Pa<^ Pp C 'P(Cj^) if and only if a = /?, and with multiplicity 
one in each of them. Therefore a linear basis of P^(CJ) is given by the polynomials 

dim(Pa) dim(Pa) 

(4.2) Pa{z,z)= J2 Mz)M^= Yl ' 

h=l h=l 

where {vi, . . . , fdim(Pa)} is any orthonormal basis of Pa in the jF^-norm. 

A result in [12] ensures that there exist 6i, . . . ,6d in A such that the polynomials 

7i =PSj j = l,...,d, 

freely generate V^{C^). In [8] the authors prove that 71, ... ,7^ have rational coefficients. 
More precisely, setting rrij = \6j\, each 7j can be written in the form 

\a\ = \h\=mj 

where a, b are in and 6^]^ are rational numbers. 

The symmetrization L^. of 'jjiZ, Z) is a homogenous operator of degree 2mj in Di^ with 
rational coefficients, and {—iT, L^-^, . . . , L^^} generate [U E]. Moreover, the eigenvalues 
-^7j (0a) are rational numbers 012]. 

Fix any positive integer m and denote by Mj^m the matrix which represents the restriction 
of dni (L-y.) to Vm{C") in the basis of monomials w"", \a\ = m. For every F in jFi(C"), 

[d7c,{Zh)F]{w) = [d^,F]{w) [d7i,{Zh)F]{w) = -^WhF{w) \/w G C", 

for h = l,...,n. Therefore Mj^m has rational entries, with denominators varying in a 
finite set independent of m. We can then take such that the matrices NMj^m have 
integral entries, for all m and j = 1, . . . ,d. Thus the characteristic polynomial of NMj^m is 
monic, with integral coefficients and rational zeroes NLy.{(f)a)', therefore these zeroes must 
be integers. They all have the same sign, independently of m, equal to (— l)'"^ (cf. [U]). 
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We then define 

V; = iV(-l)-^L,^+£™^ J = l,...,d, 

where C = —2J2]=i{ZjZj + ZjZj) is the U(ri)-invariant sublaplacian, satisfying jC{(l)a) = 
2\a\ + n. We show that {Vq = —iT, Vi, . . . , V^} is a set of generators satisfying the required 
conditions. 

Since Em,=i7j = (cf- iSj), we have 



" — — 1 



and then 



(4,3) 



mi=l m,,=l ^ 



where r is the cardinality of the set {6j : rrij = \Sj\ = 1}. Therefore each L^. is a polynomial 
in Vi, . . . , Vd- Since — ^T, L^^, . . . , L^^ generate D;^, the same holds for —iT, Vi, . . . , V^. 



Condition (IT| follows from the homogeneity of Ly. and /^'"j , and from (4.2). Since the 



polynomials in (4.2) are real- valued, then the V's are formally self-adjoint and condition (\2 



is easily verified. Finally condition ([s]) follows from (2.2), which gives 



for all w in C"' and j = 1, . . . ,d. □ 



Let V = {Vo, Vi, . . . , Vd} denote the privileged set of generators chosen in Lemma 4.1 We 
set p = (pi, . . . , pd) the polynomial map in (|3| of Lemma 4.1 



Coordinates in M'^+i will be denoted by (A, 0, with A in M and ^ in R'^. So, if (A, ^ = V{(f)), 
then either A = —iT{(j)) = 0, in which case (j) = rjKw and ^ = p(w, w), or A = —iT{(f)) ^ 0, in 
which case = 0A,a and = Vj{(j)\^a) = |A|™^ Vj{(j)a)- 

The spectrum consists therefore of two parts. The first part is Sq = {0} x p(C"), a 
semi-algebraic set. The second part, S', is the countable union of the curves ro,(A) = V(0A,a)) 
A 7^ 0. Each Fq and Sq are homogeneous with respect to the dilations 

(4.4) (A, ei. . . . , ^d) ^ (iA, t^^ei- • • • , f^'^d) , (t > 0) 
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and to the symmetry (A,^i. . . . ^ ■ ■ ■ y^d)- By our choice of V, S' fl {A = 1} is 

contained in the positive integer lattice. Moreover E' is dense in (cf. [Gj [T5] ) 

For the sake of brevity, we denote by / the Gelfand transform Q^f of the integrable 
i^-invariant function / . 



5. Functional calculus 



In this section we prove one of the two imphcations of Theorem 1.1 for a closed connected 



subgroup K of U(n) such that {K x if„, K) is a Gelfand pair. More precisely we prove 
that if m is a Schwartz function on M'^"''^, its restriction to S)^ is the Gelfand transform of a 



function / in Sx^Hn) (see Theorem 5.5 below). 



The proof is based on a result of Hulanicki [20] (see Theorem 5.1 below) on functional 
calculus for Rockland operators on graded groups and a multi-variate extention of it. 

A Rockland operator D on a graded Lie group is a formally self-adjoint left-invariant 
differential operator on which is homogeneous with respect to the dilations and such that, 
for every nontrivial irreducible representation vr of A^, the operator dTT{D) is injective on the 
space of C°° vectors. 

As noted in [21], it follows from [25] and [16] that a Rockland operator D is essentially 
self-adjoint on the Schwartz space S{N), as well as dTT{D) on the Garding space for every 
unitary representation tt. We keep the same symbols for the self-adjoint extensions of such 
operators. 

Let A^ be a graded Lie group, and let | ■ | be a homogeneous gauge on it. We say that 
a function on A^ is Schwartz if and only if it is represented by a Schwartz function on the 
Lie algebra n in any given set of canonical coordinates. The fact that changes of canonical 
coordinates are expressed by polynomials makes this condition independent of the choice of 
the coordinates. Given a homogeneous basis {Xi, . . . of the Lie algebra n we keep the 
same notation Xj for the associated left-invariant vector fields on A^. Following [12], we shall 
consider the following family of norms on S{N), parametrized by a nonnegative integer p: 

(5.1) ||/||(p,^) = sup{(l + |a;|)^'|XV(x)| : x e N , degX'<p}, 
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where = Xl^ ■ ■ ■ X^ and degX^ = ^ z^degX^. Note that the Frechet space structure 
induced on S{N) by this family of norms is independent of the choice of the Xj and is 
equivalent to that induced from S{n) via composition with the exponential map. 

Theorem 5.1. [20] Let D a positive Rockland operator on a graded Lie group N and let 
D = f^°° \dE{X) be its spectral decomposition. If m is in SiM.) and 

r+oo 

m{D) = / m{X)dE{X), 
Jo 

then there exists M in S{N) such that 

m{D)f = f*M WfeS{N). 
Moreover for every p there exists q such that 

Suppose that Di, . . . , Dg form a commutative family of self-adjoint operators on (in 
the sense that they have commuting spectral resolutions). Then they admit a joint spectral 
resolution and one can define the bounded operator m{Di, . . . , Ds) for any bounded Borel 
function m on their joint spectrum in M*. The following theorem was proved in [27] in a 
special situation. 

Theorem 5.2. Suppose that N is a graded Lie group and Di, . . . ,Ds form a commutative 
family of positive Rockland operators on N. If m is in S{W), then there exists M in S{N) 
such that 

m{D,,...,D,)f = f*M. 

Moreover, for every p there exists q such that 

\\M\\(^p^N) < C 



Proof. We prove the theorem by induction on s. By Theorem 5.1 the thesis holds when s = 1. 
Let s >2 and suppose that the thesis holds for s — 1. Let m(Ai, . . . , As) be in iS(]R*). Then 
there exist sequences {^ipk} in iS(]R''~^) and {fk} in S(M.) such that m(Ai, . . . , A^-i, As) = 
Y.ki^k{Xi, Xs-i)ipk{Xs) and Y.k 11^^ ® V^fclUiR« < for every A^. 
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For a proof of this, one can first decompose m as a sum of C°°-functions supported in a 
sequence of increasing balls and with rapidly decaying Schwartz norms, and then separate 
variables in each of them by a Fourier series expansion (see also [27] ) . 

By the inductive hypothesis, for every k there exist and in <S{N) such that 
MDi, . . . , D,_i)f = f*^k and MDs)f = f*^k for every / in S{N). Then 
ipk ® <^k (Di, • • • , -D^-i, Ds) f = i^k (-Di, . . . , Ds-i) ipk (Ds) f 

= / * $fc * . 

By straightforward computations (cf. [Ill Proposition 1.47]) and the inductive hypothesis, 
we obtain that 

ll^fc * ^k\\(p,N) < Cp W'^kWip^N) ||*^'fc||(p"+l,Af) 

< Cp ||^fc||(g,K^-l)||v^fe||{g',M) 

< Cp WiJk ® V^fc||(g",E'') • 

Since the series Ylk'^k ® V'fc is totally convergent in every Schwartz norm on W, there 
exists a function F in S{N) such that 

k 

and hence 

J2f *'^k*^k = f * F 

k 

for every / in S{N). 

On the other hand, if M G S'{N) is the convolution kernel of m {Di, . . . , Dg-i, Dg), by 
the Spectral Theorem, 

m (A, . . . , Dd-u Dd) f = J2^kiDi,..., Dd-i) ® Vk (Dd) f 

k 
k 

for every / in S{N), with convergence in L?'{N). Therefore ^k * ^k converges to M in 
S'{N), i.e. F = M and M is in S{N). 
Finally, given p there exists q such that 
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This follows from the Closed Graph Theorem. Indeed, we have shown that there is a linear 
correspondence m i— >■ M from iS(]R'^) to S{N); moreover, reasoning as before, if rrih — m in 
S(R'^) and Mh ^ if in S{N) then Mh ^ M hj the Spectral Theorem in S'{N). Therefore 
ip = M. □ 

Going back to our case, we prove that the operators Vi, . . . ,Vd satisfy the hypotheses of 
Theorem I 



Lemma 5.3. The differential operators Vi, . . . ,Vd defined in Lemma 4-i form a commutative 
family of positive Rockland operators on 

Proof. Suppose that j = 1, . . . ,d. The formal self-adjointness of Vj follows directly from its 
definition and the identity Zkfg = ^ Jh f ^kd, for every pair of Schwartz functions / 
and g on 

We check now the injectivity condition on the image of Vj in the nontrivial irreducible 
unitary representations of 

By ^ in Lemma [ilj dT^{Vj) = pj{w) > for w 7^ 0. 

As to the Bargmann representations ttx, the Garding space in J-'\x\ can be characterized as 
the space of those F = ^^g^ Fa (with F^ in Pa for A > and in Pa' for A < 0) such that 

j2i^+\a\r\\Far^^^^<oo 

for every integer A^. For such an F, 

V,F = Y,VMx,a)Fa 

where the series is convergent in norm, and therefore it is zero if and only if = for every 
a. 

Positivity of Vj follows from Plancherel's formula: for / G S{Hn), 
since the eigenvalues V^(0A,a) are positive. 
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Given a Borel subset uj of IR+, define the operator Ej{uj) on L'^{Hn) by 
(5.2) vr,(E,H/) = J2xu.{Vj{M))7rx{mx,o. , 

wfiere Xui is the characteristic function of u and Hx^a is the orthogonal projection of J^\x\ onto 
Pq if A > 0, or onto Pa' if A < 0. Then Ej = {Ej{u)} defines, for each j, a resolution of the 
identity, and, for / G S{Hn), 

r+oo 

/ ^dE,{Of = V,f. 
Jo 

Therefore Ej is the spectral resolution of the self-adjoint extension of Vj. It is then clear 
that Ej{ijj) and Ek{uj') commute for every oj.uj' and j, k. □ 



Corollary 5.4. Let Vq, . . . , he the differential operators defined in Lemma \4-1\ If m is in 

S(R'^^^), then there exists M in SK{Hn) such that 

m(H, ...,Vd)f = f*M V/ e S{Hn). 

Moreover, for every p there exists q such that 



Proof. We replace Vq = —iT by Vq = —iT + 2C By (4.3), Vq is a linear combination of the 
Vj. Therefore m(Vo, . . . , V^) = fh{Vo, . . . , V^), where rh is the composition of m with a linear 
transformation of ]R'^+^. 

Moreover, Vq is a positive Rockland operator (cf. [13j), which commutes with the other 



Vj because so do Vq and C Applying Theorem 5.2 

m{Vo,...,V,)f = f*M, 

with M e SK{Hn) and 

< C ||m||(^^]fjd+i) < C ||m||(g^]fjd+i) . 

□ 

Theorem 5.5. Suppose thatm is a Schwartz function on W^^^ . Then there exists a function 
M in SxiHri) such that M = m\ y . Moreover the map m i — > M is a continuous linear 
operator from 5(]R'^^^) to SxiHu)- 
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Proof. It follows from (5.2) that the joint spectrum of Vq, . . . , is (cf. [HI Theorem 



1.7.10]). Therefore the continuous map m i— > M of Corollary 5.4 passes to the quotient 
modulo {m : m = on Sj^}. 



On the other hand, by (5.2) 



and this must coincide with TTx{f)TTx{M). It follows that M(y{(f>x^a)) = fn{V{(px,a)) for 
every A, a. By density, M = m\ ^ . □ 



6. Extension of Schwartz invariant functions on 

Suppose that K isa compact Lie group acting orthogonally on M™. It follows from Hilbert's 
Basis Theorem [28] that the algebra of if-invariant polynomials on is finitely generated. 
Let pi, . . . , prf be a set of generators and denote by p = (pi, . . . , p^) the corresponding map 
from to R'^. The image S = p(M™) of p is closed in M^. 

If /i is a smooth function on M'^, then /i op is in C^(]R'^), the space of fC-invariant smooth 
functions on M."^. G. Schwarz [2S] proved that the map h ho p is surjective from C°°(]R'^) 
to C^(]R™'), so that, passing to the quotient, it establishes an isomorphism between C°^(S) 
and 

J. Mather [21] proved that the map h ho p is split-surjective, i.e. there is a continuous 
linear operator S : C^(M'^) -> C°°(M^) such that (Sf) o p = / for every / e C^(M'"). 

From this one can derive the following analogue of the Schwarz-Mather theorem for 
5k(M'"). 

Theorem 6.1. There is a continuous linear operator £' : SxiW^) ^ S{W^) such that 
{£' g) o p = g for every g G iS/^(]R™). In particular, the map g g o p is an isomorphism 
between S{J:) andSKiR""). 



Proof. It follows from Lemma that the validity of the statement is independent of the 
choice of the Hilbert basis p. We can then assume that the polynomials pj are homogeneous 
of degree aj. 
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(5,(yi,...,i/fc) = (r"^Z/i,...,r"'*l/d) Vr>0, 

and we shall denote by | ■ |q a corresponding homogeneous gauge, e.g. 

d 

(6.1) \y\o. = cJ2\yi\'^"' ' 

j=i 

satisfying \Sry\a = r\y\a- 

On M™ we keep isotropic dilations, given by scalar multiplication. Clearly, p is homoge- 
neous of degree 1 with respect to these dilations, i.e., 

p{rx) = 6r{p{x)) Vr > 0, x G M"" , 

and S is (5r-invariant for all r > 0. 

Since p is continuous, the image under p of the unit sphere in is a compact set not 
containing (in fact, since is a polynomial in the pj, cf. [24j, Pj{x) = for every j 
implies that a; = 0). 



Choosing the constant c in (6.1) appropriately, we can assume that 1 < |p(x)|q, < R for 



every x in the unit sphere in M™. It follows by homogeneity that for every a,b, < a < b, 
(6.2) p{{x:a< \x\ < b}) C {y : a < < Rb}. 

Fix S : C'^{W^)x C°^(]R'') a continuous linear operator satisfying the condition {Sf) o 
p = f, whose existence is guaranteed by Mather's theorem. 

Denote by Bs the subset of M"' where \y\a < s. For every p & N there is g G such that, 
for / supported in the unit ball, 

\\^f\\cHB,.)<C,\\f\\c. ■ 

Given r > 0, set fr{x) = f{rx) and 
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By the homogeneity of p, {Srf) o p = f. For r > 1 and / supported on the ball of radius 
r, we have 



(6.3) II^JIIc.(B,,.)<C,r«||/||c. . 

Let {fj}j>o be a partition of unity on consisting of radial smooth functions such that 

(a) (fo is supported on {x : \x\ < 1}; 

(6) for j > 1, ipj is supported on {x : R^~^ < \x\ < R^; 

(c) for j > 1, ipj{x) = ipi{R-^^'^h). 

Similarly, let {il'j}j>o be a partition of unity on M'^ consisting of smooth functions such 
that 

(a') ipo is supported on {y : \y\a < R}; 

{b') for j > 1, ipj is supported on : R^~^ < \y\a < R-'^^}', 

(c') for j > 1, ^j{y) = tlJi{6R-o-i)y). 

For / e 5i^(M"') define 

oo 1 oo 1 

j=0 l=-2 j=0 e=-2 

with the convention that ip^i = ip^2 = f-i = 0. Then 

oo 1 
j=0 i=-2 



By (6.2), J2l=-2 = 1 on the support of (pj, hence S' f o p = f. 



We have the following estimate for the Schwartz norms in (5.1): 



3=0 



On M'^ we adapt the Schwartz norms to the dilations 6r by setting 

\9\\[p,R<i)= ^ sup {l + \y\c,Y\d''g{y)\ . 
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We then have 
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i=o 



ill CP 



J=0 



Therefore 



j=o e=-2 

oo 1 



j=0 i=-2 
oo 1 

j=o e=-2 

oo 

cJ2R'l^R^ivjf)\\ CP(B„j+2) ■ 

j=0 



By (6.3), since ipjf is supported on the ball of radius , 

oo 

\\£' f\\[p,Rr^) < Cp^R^^^^''^\\Vjf\\ci < Cpli; ||(p+g,IR.n) 



j=0 



□ 



7. Schwartz extensions of the Gelfand transform of / g SK{Hn) 
In this section we suppose that i^' is a closed connected subgroup of U(?7.). The following 



theorem settles the proof of Theorem 1.1 in this case. 



Theorem 7.1. Let f be in SK{Hn)- For every p inN there exist Fp in iS(]R'^"^^) and q in N, 
both depending on p, such that Fp, ^ = f and ||Fp||. ,^^+1) < Cp 

Notice that this statement implies the existence of a continuous map from iS(S]^) to 
SxiHn) that inverts the Gelfand transform, even though its formulation is much weaker 



than that of Theorem |6.1[ We do not claim that for each / a single F can be found, all of 
whose Schwartz norms are controlled by those of /. In addition, our proof does not show if 
Fp can be chosen to be linearly dependent on /. 



The proof of Theorem 7.1 is modelled on that given in [T] for the cases K = U(n),T"', 
but with some relevant differences. On one hand we present a simplification of the argument 



22 F. ASTENGO, B. DI BLASIO, F. RICCI 

given there, disregarding the partial results concerning extensions of / with finite orders of 
regularity; on the other hand extra arguments are required in the general setting. 

We need to show that the Gelfand transform / of / G SxiHn) extends from E)^ to a 
Schwartz function on M'^^^. Our starting point is the construction of a Schwartz extension 
to all of {0} X M'^ of the restriction of / to 

So = {VivK^ ■.weC^} = {0} X p(C") . 

If JF/ denotes the Fourier transform in C" x M, 

^/(A, w)= [ f{t, ^)e-*(^*+R«--) dw dt , 

we denote 

f{w)=J^fiO,-w) = f{0,piw)) . 

To begin with, we set 

(7.1) f = £'feSiR') . 

Then /\0 = /(0,0 if (0,OeSo. 

The next step consists in producing a Taylor development of / at A = 0. The following 
result is derived from [15]. In our setting the formula must take into account the extended 



functions in (7.1 ). 



Proposition 7.2. Let f be in SxiHri)- Then there exist functions fj, j > 1, in SkIH^), 
depending linearly and continuously on f , such that for any p in N, 

/(^' = E 7! (0 + T^VW 0, V(A, e ^K, 

j=o 



where fo = f and fj " is obtained from fj applying (|7.1|). 



Proof. For / in SxiHu), we claim that the restriction of u{X,^) = f (^) to is in iS(Sj^). 
It is quite obvious that u is smooth. Let ip he a, smooth function on the line, equal to 1 on 
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[—2,2] and supported on [—3,3]. Define 



By (4.1) and rt2j) in Lemma 4.1, \1/ is equal to 1 on a neighborhood of S^. It is also 



homogeneous of degree with respect to the dilations (4.4) outside of a compact set. Then 



"^u is in iS(]R'^"*""^) and coincides with u on 



K- 



It follows from Corollary 5.4 that there exists h in SxiHri) such that 

/(A,0-/\0 = /^(A,0 V(A,OeS]^. 
Since h(0, p{w)) = for every w, h{z, t) dt = for every z. Therefore 

fi{z,t) = / h{z, s) ds 

J — oo 

is in SxiHn) and 

MA,0 = A/i(A,0 V(A,OeS^. 

It easy to verify that the map U : / i-^ /i is linear and continuous on SxiHri)- We then 
define /j, j > 1, by the recursion formula fj = jUfj^i and the thesis follows by induction. □ 

We use now the Whitney Extension Theorem |23] to extend the C°°-jet {d^fj }(j^a)em+i 
to a Schwartz function on 1^"^+^. In doing so, we must keep accurate control of the Schwartz 
norms. For this purpose we use Lemma 4.1 in [Ij, which reads as follows. 

Lemma 7.3. Let k >1 and let h(\,^) be a C'^-function on x M" such that 

(1) 9^/i(0,O = for \a\ <k andie M"; 

(2) for every p G N, 

a,{h)= sup \\{l + \-\Yd'^dlh\\^<^ . 

\a\+m<k 

Then, for every e > and M G N, there exists a function H^^m G iS(]R™' x M") such that 

(1) 9^/ie,M(0,O = for every aeN"" and^e M"; 

(2) sup|,|+i^i<,_i 11(1 + 1 ■ \rd^d^^{h - h,,M)\L < ^/ 
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(3) for every p eN there is a constant Ck,p,M such that 

\\heM\{pM^+-) < ^fc,P,M(l + aM{hYe-P) ||(1 + | ■ \Yh\\^ . 

The following proposition is in [H Proposition 4.2] for K = T". We give here a simplified 
proof. 

Proposition 7.4. Given f E SK{Hn) and p G N, there are H E S(R.'^~^^) and q E N such 
thatdiH{0,O = fAO and 

||-f^||(p,IRd+l) < Cp\\f\\iq,H„) ■ 

Proof. Let rj he a smooth function on M such that r]{t) = 1 if |t| < 1 and rjit) = if \t\ > 2. 



By Theorem 6.1 and Proposition 7.2, for every k and r there exists qk^r such that 



(7.2) 



ll/fc ||(r,IR'') < Ck,r\\f\\{qk,r,Hn) ■ 



We fix p G N and apply Lemma 7.3 to 



Hypothesis (1) is obviously satisfied and (2) also, because hk is a Schwartz function. By 



(7.2) 



(7.3) 



ar{hk) < Ck,r\\f\\{qk + l,r,Hr.) 



Let q be the maximum among the qk^p for k < p + 1. Setting Ek 
for each k there is a function Hk E 5(]R'^^^) such that 

(i) diHkiO, 0=0 for all j G N and ^ G R'^; 

(ii) sup|„|+|^|<,_i 11(1 + 1 ■ lyd-.d^^ihk - Hk 



{q,H„), M =p, 



(iii) for k < p, using (7.3) 



Define 



< Ck,p{l + e~klf\\lj,J\\{l + I ■ lYhkW^ < . 



H = Y,hk-J2Hk+ J2 (hk-Hk) . 

k=Q k=0 k=p+l 
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By (7.2), (ii) and (iii), the p-th. Schwartz norm of H is finite and controlled by a constant 



times the g-th Schwartz norm of /. Differentiating term by term, using (i) and the identity 
dihk{0,0 = Sj,k+ifk+i (0> we obtain that d{H{0,^) = J (0 for every j. □ 

Let now be a smooth function on M such that ip{t) = 1 if \t\ < 1/2 and (p{t) = if 
\t\ > 3/4. For h defined on S)^, we define the function Eh on ]R"'+-'^ by 
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\X\mi 



VA 



where ^(A.a) = (t4(0A,a), • • • , VdiM) = (|Ar^t7i(0j 
Recall that, by Lemma 



lA 



A = 0, 



4.1 



each V£(0q) is a positive integer. Therefore, if < for some 
every term in the series vanishes, whereas, if ^ is in M^jJ., the series reduces to at most one 
single term. Moreover, for every g in S{Hn), E'g = ^ on S'. 

The proof of the following result goes as for [H Lemma 3.1], using P, p. 407] instead 
of [H (2.2)]. In contrast with |1] we state it only for vanishing of infinite order of the Taylor 
development of the Gelfand transform on Eq. 

Proposition 7.5. Suppose that g in SxiHn) and gj^^ = for every j . Then 

(1) Eg{X, = ?(A, for all (A, G ^l; 

(2) di{Eg){0,0 = for all s and ^ e M^' 

(3) for every p > there exist a constant Cp and an integer g > such that 



(p,Md+i) < Cp\\g\\(^q^H„) 



In particular, E'g G i5(]R^ ) 



To conclude the proof of Theorem 7.1 , take / in SxiHri) and p in N. Let H be the function 
m depending on p, defined as in Proposition 7.4 By Theorem 5.2 there exists h 

in S{Hn) such that 
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Define 



F = E{f - h) + H 



and the thesis follows easily. 



8. General compact groups of automorphisms of H, 



n 



We have discussed in the previous sections the Gelfand pairs associated with connected 
subgroups of U(n). In this section we only assume that if is a compact group of automor- 
phisms of Hn- Let Kq be the connected identity component of K. Then Kq is a normal 
subgroup of K and F = K/Kq is a finite group. Conjugating K with an automorphism if 
necessary, we may suppose that Kq is a subgroup of U(n). 

For D in D^q and w = kK^ in F, define by 



Since Kq is normal, D"^ is in I^Ko- It is also clear that I^Kq admits a generating set which 
is stable under the action of the group F. Indeed, it suffices to add to any given system of 
generators the F-images of its elements. Denoting by V the linear span of these generators, 
F acts linearly on V. Let V = {Vi, . . . , V^} be a basis of V, orthonormal with respect to an 
F-invariant scalar product. Clearly, is a generating set for Hkq- 

Applying Hilbert's Basis Theorem as in Section [6| there exists a finite number of (homo- 
geneous) polynomials pi, . . . ,pr generating the subalgebra Pp(]R'^) of F-invariant elements 
in P(]R'^). Let p = {pi,...,pr) : M"' ^ M*" be the corresponding Hilbert map and let 



When / is ii'o-invariant and w = kK^ in F, we set f o w = f o k. 

Lemma 8.1. The set W = {Wi, , . . . , W^} generates D^-. Moreover if ip is a K-spherical 
function, then 



D'"f = D{f ok-^ 



o k 



W,=p,{y,,...,Vd)ioT3 = l,...,r. 



(8.1) 




for some Ko-spherical function 0. 
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Proof. Take D in Dx- As an element of Dj^^, D is a polynomial in the Vj. Averaging over 
the action of F, we can express D as an F-invariant polynomial in the Vj. Hence D is a 
polynomial in Wi = piiVi, . . . ,Vd), . . . ,Wr = pr{Vi, Vd). 

Recall that all Kq- and i^-spherical functions are of positive type [3]. Let Vk (resp. Vkq) 
denote the convex set of i^-invariant (resp. ii'o-invariant) functions of positive type equal 
to 1 at the identity element, and consider the linear map J : LJ^^^ —>■ defined by Jyj = 
1^ Yliw&F f^'w. Since Vkq and Vk are weak*-compact and J maps Vko to Vk, the extremal 
points of Vk are images of extremal points of Vkq- This proves that every i^-spherical 



function has the form (8.1). 



Conversely, if ip is given by (8.1) and D G D^^, then 



showing that ip is ii'-spherical. □ 



From Lemma 8.1 we derive the following property of the Gelfand spectra: 



p(sl^j = Sir C 

If V is as above, the linear action of F on V leaves invariant. For a i^To-invariant function 
/ and w in F, 

(8.2) gy{fow) = {gvf)ow 

Let Spi^^g) be the space of F-invariant elements in iS(S^^). 

Lemma 8.2. The map f ^ f o p is an isomorphism between S{T}^) and 5i?(S^^). 

Proof. If / is in iS(S^), let / be any Schwartz extension of / to W . Then (7 = / o p is an 
F-invariant Schwartz function on and its restriction to is / o p. This proves the 
continuity of the map. 

Conversely, given g in iSf(S)^^), let g be an F-invariant Schwartz extension of g to W^. 
Set h = {S'g)\ ^, where S' is the operator of Theorem 6.1 for the group F. The proof that 



the dependence of h on g is continuous is based on the simple observation that, for any 
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Schwartz norm || ||(7V) on M"', the infimum of the norms of all extensions of g is the same as 
the infimum restricted to its F-invariant extensions. □ 



We can now prove Theorem 1.1 for general K. Assume that is a compact group of 
automorphisms of if„ and let Kq, F, Vi, . . . , V^, p be as above. 

Take / in L]^{Hn). Denote by Qyf (resp. Qwf) its Gelfand tranform as a i^'o^^variant 
(resp. i^'-invariant) function. Then Qyf = Qwf ° P- In particular, a ii'o-invariant function 
is if-invariant if and only if Qyf is F-invariant. 

If / is in SxiHn), then / is also f^Q-invariant and Qyf is in Spi^^^) by Theorem 



7.1 



therefore Qwf is in iS(Sj^) by Lemma |8. 2 



Conversely, if Qwf is in it follows as before that Qyf is in Sp{Tj^ ) and therefore 



/is in Sk{H^) by (8.2). 
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